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Abstract. We study the learning dynamics of agents who adapt to heterogeneous 
comfort levels in the context of an El-Farol type game, and show that even an 
infinitesimal degree of heterogeneity in the resource levels leads to a significant 
Q , reduction of the fluctuations of the collective action, and removes the phase transition 

C/3 I observed in models with homogeneous comfort level. Our analysis is based on 

C/2 ' dynamical methods of disordered systems theory, in particular on a generating 

. ^ , functional approach, and confirmed by numerical experiments. We also report on 

simulations of a system in which the comfort levels fluctuate in time, and point 
(~| ■ out crucial differences between models in which the comfort levels of the agents 

Oh| fluctuate collectively and individually respectively. Finally we comment on a possible 

characterisation of El-Farol and Minority Games according to the presence or absence 
of ergodicity-breaking phase transitions at infinite integrated response. 
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, global behaviour, which cannot be explained straightforwardly from their microscopic 
^ interactions. The tools of statistical mechanics have here been seen to be able to 

provide further insight, and allow for theoretical progress for a variety of agent-based 
models. Most notably, analytical solutions for the Minority Game (MG) , a mathematical 
abstraction of the El-Farol bar problem, have been obtained using equilibrium and non- 
equilibrium techniques of disordered systems theory [U El E! • 

In the El-Farol bar problem = 100 customers have to decide independently 

whether or not to attend a concert in the bar, the latter having a capacity of A = 60 
seats. Agents will in general not enjoy the bar if it is too crowded, i.e. if more than 
A agents attend. Thus depending on the attendance A, agents who go to the bar are 
considered winners if A < A, and conversely, agents who decide to stay home win if 
A > X. We will refer to A as the comfort level in the following. Players in the El-Farol 
bar problem are inductive agents: they use individual 'predictors', based on the past 
attendance, to predict whether or not the bar will be crowded at the next time step, 



Complex adaptive systems of inductive agents often display a remarkably rich 
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and according to these predictions they then decide whether to attend or not. They 
learn by experience and employ the most accurate predictor which they can access. 

Under fairly weak assumptions it can be shown that the temporal average of the 
attendance, (A), converges to the comfort level A in Arthur's El- Parol model provided 
the predictors are not systematically biased ^ . A second question relates to the degree 
to which agents are able to reduce the fluctuations of A about A, i.e. to the quantity 
cr^ = (y4^) — (A)'^, where the brackets denote an average over time. We will refer to o"^ 
as the volatility in the following. 

The MG in its original form ^ is a mathematical formulation of the El-Farol 
problem at comfort level A = All agents take binary decisions at each time-step, 

and players in the minority group win. Predictors, fixed at the beginning of the game 
and randomly generated, are here unbiased in the sense that they advise to attend or stay 
home with equal statistical weights. Generalisations to A 7^ A^/2 and/or systematically 
biased strategies are possible, and have been studied by numerical simulations in jHllZI. 
More systematic studies of El-Farol games with general uniform comfort levels have 
been presented 0, see also p. 

In this paper we consider an El-Farol type problem with heterogeneous comfort 
levels, so that each agent i prefers to attend the bar only HA < Aj, where A, varies 
across the population of agents. Our work is an extension of [HI, and we reproduce 
some results reported there as a special case. We here employ a dynamical approach 
whereas the analysis of is based on static replica calculations. Both approaches are 
now standard in the context of MGs and related models [Hl^. 

The purpose of this study is here twofold. Firstly we aim at understanding the role 
of heterogeneous comfort levels for the learning dynamics of the agents. In particular 
we will be concerned with the question of whether or not heterogeneity in the comfort 
levels compromises the ability of the population of agents to converge to a 'mean' comfort 
level. Similarly we will study the influence of heterogeneity in the comfort levels on the 
temporal fluctuations of the attendance. The second reason for studying the present 
system rests in more theoretical issues related to the phase behaviour of MGs and related 
models. In the standard MG a phase transition between an ergodic and a non-ergodic 
regime has been found, and identified with a point in parameter space in which the 
integral over the response function of the system becomes singular [3 E! • In the non- 
ergodic regime a continuum of attractors of the learning dynamics appears to be present 
and introduces interesting effects such as a marked sensitivity to initial conditions, which 
is absent in the ergodic phase. 

Some variants of the MG, however, which are typically only slight modifications of 
the original game have been seen not to display the above transition. Some variants 
exhibit no transition at all, others one that is of a different type than the one described 
above. The transition is absent for example in so-called grand-canonical MGs (GCMG) 
[TU] . and similarly such an absence may be suspected for MG models with finite score 
memories ^T] although no analytical results are available as yet. In MGs with impact 
correction ^21 and dilution of the agents' interaction matrix jT2| the transition of the 
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type as in the original MG is preceded by one which is referred to as 'memory-onset' 
transition Ergodicity breaking here still occurs, but at finite integrated response. 
This type of transition has been associated with replica-symmetry breaking in static 
studies. The second purpose of this paper is thus to try to shed some more light on the 
circumstances under which MG models display a transition of the type of the original 
MG. We here show that MGs with any degree of heterogeneity in the comfort levels 
do not display this type of transition, and conclude with some conjectures on how the 
structure of the agents' learning dynamics may determine the presence or otherwise of 
such a transition. 

2. Definition of the model 

We consider a system of agents, labelled with Roman indices i,j E {1, . . . , N}. At 
each round t of the game each agent i takes a binary decision bi{t) E { — 1,1} in response 
to the observation of a common piece of publicly available information, labelled by fi{t). 
bi{t) = 1 may here correspond to player i attending the bar at time t, and bi{t) = — 1 to 
him not attending. While the information encodes the actual history (i.e. the previous 
attendances) in the original version of the El-Farol problem and in the original MG, 
we will here assume that ^{t) is chosen randomly and independently from a set with 
P = aN possible values at any t, i.e. G {1, . . . , aN}. This has been seen not to 
alter the qualitative behaviour of different variants of the MG PQEj- a = P/N is here 
the main control parameter of the model, and taken not to scale with A^. One then 
defines the re-scaled total outcome at round t as A(t) = N^^^"^ J^ibiit), resulting in 
-y/N < A{t) < y/N (in steps of 2/y/N). Note that in this notation {bi{t) E {-1, 1} as 
opposed to bi{t) E {0, 1} for not-attending/attending the bar) an attendance of N/2 in 
the original El-Farol problem corresponds to A = 0, and accordingly the comfort level 
in an actual MG (in which the bar is considered crowded if more than precisely half of 
the agents attends) would be A = 0. Agents who prefer a bar filled by less than half 
have Aj < 0, agents who can enjoy the bar even if it is populated by more than N/2 
attendees have Aj > 0. In general the attendance in the original El-Farol problem can 
be obtained as {\/N A{t) + N) / 2 from the global action A{t), in our conventions. Having 
this simple linear transformation in mind, we will refer to A{t) as the attendance at time 
t in the following, and, as mentioned above, to Aj as the comfort level of player i. Note 
that A{t) remains a well-defined finite quantity in the thermodynamic limit A^ ^ oo at 
a = P/N finite, with which the statistical mechanics analysis of the model is concerned. 

In order to take their decisions on whether or not to attend each agent i holds two 
fixed trading strategies (look-up tables) Rj^a = {Rla: • • • > -^fa) ^ {~1> 1}^> with a = ±1. 
The binary entries are drawn from some distribution (to be specified below) before 
the start of the game; these entries (along with the comfort levels Aj) represent the 
quenched disorder of this problem. If agent i decides to use strategy a in round t of the 
game, his action at this stage will be bi{t) = -Rf^*''. The agents decide which strategy to 
use based on points Pi,a(t) which they allocate to each of their strategies. These virtual 
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scores are based on the player's potential success had he always played that particular 
strategy, and are updated as follows at every time-step: 

Strategies which would have produced a decision to the liking of player i are thus 
rewarded, i.e. if A{t) > Aj the scores of strategies a with i?fi*^ < are increased and 
vice versa for A{t) < Aj. In the MG literature the above type of process (P) with an 
explicit dependence on the influx of information patterns fi{t) is generally referred to as 
'on-line dynamics'. At each round t each player i then uses the strategy in his arsenal 
with the highest score, i.e. bi{t) = i?^^*!^^, where Si{t) = arg max^pia{t). It remains to 
specify the statistics of the {Ria}- We here choose 

P{R^,a) = ^ + ^ (2) 

as the distribution from which the -Rf„ are drawn (with 6x,y = 1 if x = y and 6x,y = 



otherwise), so that R^^ = pi/VN for all a,fi {'^ here denotes an average over the 
quenched disorder). In the limit of large A^, which we will eventually consider, the 
distribution of the i^f^ is well-defined for any pi G M. If pi > then player i's 
strategies are more likely to advise him to take a positive action (6j = +1) than a 
negative one, and vice versa. We will refer to i's strategies as being consistent with his 
comfort level Aj if pi = Xi, and as inconsistent otherwise. For an agent with consistent 
strategies the probability of him attending the bar upon randomly choosing one of his 
strategies (regardless of their scores) corresponds to his comfort level. The scaling of the 
comfort levels and strategy biases with N has here been chosen to ensure a well-defined 
thermodynamic limit, and corresponds to the choices of jH|. 

In the following we will consider a population of agents in which all pi are drawn 
independently from a fixed distribution Q{p), and the Aj from R{\). The standard MG 
is recovered for Aj = pj = Vi, i.e. Q{p) = S{p) and -R(A) = As we will see 

in the further analysis, the specific details of the distribution Q{p) are unimportant, 
and the only feature relevant for the properties of the model is the mean strategy 
bias Po = / dpQ{p)p. We will also demonstrate that the model is invariant against 
simultaneous uniform shifts of the pi and Aj, i.e. under Aj — > Aj + A and pj — p^ + A 
for any A G M. Without loss of generality we will therefore mostly restrict to cases with 
Po = in the following. 

Finally the key observables we will study in this model are given by the deviation 
of the mean attendance from the mean comfort level Aq = / dXR{X)X, i.e. by | (A) — Ao|, 
and by the fluctuations of the attendance about its mean, i.e. by cr^ = {A"^) — (A)'^. 
{■ ■ ■) here stands for an average over time in potential stationary states of the system, 
i.e. after some sufficiently long equilibration time. To conclude this section we note that 
that the model of jH] can be recovered upon assuming homogeneous comfort levels and 
strategy biases, i.e. by setting pi = po and Xi = Aq for all i. 
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3. Statistical mechanics analysis 

We will here pursue a dynamical approach based on a generating functional analysis 
of the score update rules. We here consider the so-called batch process of the learning 
dynamics, which for the present case reads 



g,(t + l) = g,(t)--^5^e 



kfj:{^^s,{t)+u^}-x. 



N 



+ ^A.(t),(3) 



similar to that of the standard MG ^1 |2]. We have here introduced the score 
differences qi{t) = ^ {pi^i(t) — pi_i{t)) and the quantities = ^(^R^^—R^_j) and 
a;f = I {Rii + Ri-i) , using conventions which are now standard in the MG-literature. 
Sj{t) G { — 1,+!} indicates the strategy player j is using at time t and is given by 
Sj{t) = sgn[qj(t)], so that player j's action at time t reads +^jSj{t) on the occurrence 
of information pattern fi. hx- {t) is an external perturbation field, which is set to zero in 
all simulations and mostly a mathematical device added to generate response functions. 
Perturbations are assumed to be identical for agents with identical comfort level, hence 
the subscript Aj. Note also that compared to the on-line process an effective average 
over all information patterns fi has been performed at each time-step and that time has 
been re-scaled. 

The above batch process is the starting point for the dynamical analysis of the 
problem based on generating functionals. This technique is now standard in the context 
of the MG and we will only report the final outcome of the theory here, with some 
additional information in the appendix. For the further mathematical details of the 
computation in similar cases we refer to the recent textbook [2]. 

The generating functional analysis turns the Markovian problem of interacting 
agents into a self-consistent description in terms of decoupled effective agents subject to 
non-Markovian stochastic processes, and is exact in the thermodynamic limit N oo 
(at fixed a = P/N). For an effective agent with strategy bias A this process reads 

qxit + 1) = qxit) - a V(I + G)-}sx{t') + V^r^xit) + hxit), (4) 



t'<t 



where the second term on the right-hand side marks a retarded interaction in time 
and renders the process non-Markovian. sxif) is given by sxit) = sgn[gA(t)] and r]x(t) 
represents the stochasticity of this process. More precisely, r]x{t) is Gaussian noise of 
zero mean, and with temporal correlations 

{rix{t)rix{t')), = [(I + G)-^D{1 + G^)-\> + 2Uf, - 2A(/, + U) + 2\^Eu'. (5) 

Here lu' = Su' is the identity matrix and Eu' = 1 Vt,t', and (■ ■ refers to an average 
over realisations of the effective process (i.e. over realisations of the ?7a). C and G are 
the correlation and response functions of the system respectively 



Cu' = lim N-^Y ((si(t)si(t'))), = lim AT-i y ^^^^^ (6) 
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with ((■■■)) an average over potentially random initial conditions from which the 
dynamics is started. The matrix D finally is given by Dtt> = 1 + Cu' for all t, t' and we 
have 



ft = T[{l + G)-\pol + G%e (7) 



t' 



with 



G[,= hm N-'YX,^^^^. (8) 

These order parameters are then to be determined as averages over realisations of the 
effective processes and over the distribution of A 

Cw = ! dXR{\){sx{t)sx{t')\X)^ (9) 



Gw = J dXR{X)^j^^ {sxm). (10) 

G[, = I dXR{X)X-^j^^ {sxmi (11) 

(where (■ ■ ■ |A)^ is an average over realisations of the effective process restricted to 
representative agents with comfort level A). The further analysis then assumes the 
existence of an ergodic stationary state (in which correlation and response functions 
depend only on time-differences, i.e. Gu' = G{t — t') and similarly for G and G' 
and in which the integrated responses remain finite) and proceeds along the lines of 
[Ej. We will not report these further steps here, but only quote some of the resulting 
equations describing the persistent order parameters in the appendix. These time- 
independent order parameters are in the present problem given by the persistent part 
c = \miT^ooT~^ YliT<T^^'^) °f correlation function and by the integrated responses 
X = E.G(r) and x' = E.G'(r). 

The two observables which we will focus on, namely the mean attendance and the 
fiuctuations of the attendance, can be computed from the effective particle problem as 
follows. Similarly to J3] one finds that 

((A)) = /t, {{A,A,)) = ^[(I + G)-'D{1 + G^)-\, + (12) 

resulting inj 

wm) - Wmf = ^[(i + G)-'D{i + G^)-%. (13) 

Temporal averages can then be expressed in terms of persistent order parameters, and 
we have 

'l + c 



^ ' 1+X 2 



1 -c 



(14) 



.1 + X 

The former relation is exact, the second obtained within a common approximation 
following the lines of j^. 



I Note here also that {'n\{t)rj\{t')) ^ = 2{{{At — X){Ati — A))), reflecting the term A(t) — Ai in the update 
rules of the score difference of player i. 
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4. Results 

One observes from the above effective agent problem that the distribution of strategy 
biases Q{p) enters only through its first moment po as claimed above. Furthermore it 
is straightforward to check on the level of the effective process that simultaneous shifts 
of the distributions of p and A (i.e. pi ^ pi + A \/i at the same time as Aj ^ Aj + A 
with some i-independent A G R) do not effect the transients or stationary states of the 
model, and it becomes clear that strategy biases and comfort levels are fully equivalent 
in the homogeneous case as already found in [H^. This is also verified in simulations. 
Thus we can concentrate on the case po = 0- 

4-1. Heterogeneous comfort levels at consistent strategies 

We here impose consistency of the strategy biases with the individual comfort levels, i.e. 
that pi = Xi for all i. If the A, are then homogeneous over the population of agents, i.e. 
if \ = pi = pq for all i, then the problem can be seen to be equivalent to the standard 
MG with random external information Due to the invariance under uniform shifts 
in the pi and Aj this equivalence of the model with homogeneous consistent strategies 
to the MG holds irrespectively of po- 

We now turn to El-Farol games with a heterogeneous distribution of the Aj and pi, 
still maintaining consistency = pi. We here choose the simplest possible case, namely 
a bi-modal distribution 

R[X) = \[5{X-e) + 5{X + s)] (15) 

to study the effects of heterogeneity in the comfort levels on the behaviour of the model, 
with e > a model parameter measuring the degree of heterogeneity, e = reproduces 
the standard MG. 

One first realises that the mean attendance {A) is given by the mean comfort level 
Ao = / dXR{X)X = 0. In the theoretical analysis this is easily seen to be the case as 
x' = due to symmetry with respect to e ^ —e, and is also confirmed in simulations 
(not shown). The magnitude of the fluctuations of the attendance about this level is 
shown in Fig. ^ both as a function of a at fixed e (left) and as a function of e at 
fixed a (right). For e = one finds the functional dependence of on a typical of the 
well-known MG, with a minimum attained at an intermediate ac ~ 0.33 and both high- 
volatility and low- volatility branches at a < depending on initial conditions, as shown 
in the left panel of Fig. ^ for comparison, with open circles marking unbiased starts 
(?j(0) = 0), and filled circles starts from strongly biased initial conditions (|gi(0)| ^ 1). 
One finds that o"^ diverges as a — > for unbiased starts, and that cr^ — for strongly 
biases initial conditions [3 0] • For a > the starting point of the dynamics has no 
influence on the persistent order parameters in the stationary state. In the theoretical 
analysis the phase transition point separating both regimes is identified as the point at 
which the integrated response x diverges, marking the breakdown of the assumptions 
regarding ergodicity. 




Figure 1. (Colour on-line) Volatility for the game with consistent strategies {pi = 
Xi Vi) and bi-modal distribution i?(A) = [6{X + e) + 6{X — e)]/2. Left: as a function 
of a for different e = (circles), e = 1 (squares), e — 0.5 (diamonds) and e — 0.01 
(triangles). Open symbols are from simulations of the corresponding on-line games 
started from tabula rasa initial conditions qi{0) = (with N = 300 agents, averages 
over 50 samples are taken, simulations run for 50000 on-line steps (or more for small 
e)). Solid lines are from the analytical theory. Filled circles are for the standard 
MG {e = 0) from strongly biased initial conditions. Right: cr^ as a function of e at 
fixed a ~ 0.1, 0.2, 0.5 (circles, squares, triangles respectively). Symbols are simulations 
of the on-line process started from tabula rasa initial conditions, lines the analytical 
theory in the ergodic phase. Arrows point out that there are discontinuities of cr^ at 
e ^ for a < ac but not at a = 0.5 > Uc- 



The behaviour of the model with heterogeneous (but consistent) comfort levels is 
fundamentally different. For e > all curves are found to be increasing functions of a, 
and the volatility remains low at low values of a (for all values of e tested here), as shown 
in the left panel of Fig. ^ We here depict measurements from simulations (markers) 
as well as the predictions of the analytical theory (lines), and find excellent agreement. 
No significant dependence on initial conditions is found. The theoretical analysis shows 
no sign of a divergence of either x or x' so that we conclude that the ergodicity-broken 
phase of the standard MG is absent as soon as any degree of heterogeneity is added to 
the comfort levels. We have also performed an analysis along the lines of jlSj to check 
for a breakdown of weak long-term memory assumptions at finite integrated response, 
and could find no such a memory-onset transition. Thus the system is fully ergodic at 
all a > for e > 0. 

The marked difference between the models with and without heterogeneity in the 
comfort levels is demonstrated in plots of o"^ versus e at different values of a in the right 
panel of Fig. ^ For a > ac one observes a smooth dependence of o"^ on e with no 
discontinuity at e — > 0. For a < ac however, a characteristic jump occurs at e — > if 
simulations from zero initial conditions are considered. For all e 7^ cr^ remains low, 
whereas a substantially higher volatility is found at e = 0. The magnitude of the jump 




Figure 2. Phase diagram of the model with homogeneous resource level and 
homogeneous strategy bias (pi = po and = Aq for all i) in the (a, |po — -^ol) plane. 
Up to re-scaling the diagram is identical to that found in (H) by static methods. 



here depends on the choice of a < an increases as a becomes smaller. Note that a 
similar behaviour has been found in MGs with impact correction or dilution [T| 1^ I13j. 

Finally, while we have presented results only for bimodal distributions of the 
Aj we note that the theory presented here straightforwardly applies to more general 
distributions -R(A) as well. Simulations of the model with Gaussian R{X) of different 
non-zero widths e demonstrate that the phase transition is absent also in this case and 
that qualitative behaviour of the model is then similar to that of the game with a bi- 
modal distribution of the A^. Thus the specific distribution of the comfort levels seems 
to be irrelevant as far as the absence of the phase transition is concerned, and the only 
relevant factor appears to be the presence or otherwise of any type of heterogeneity. 

4.2. Inconsistent strategies 

We here consider the case in which Aj 7^ pj. The case of uniformly inconsistent strategies 
has been studied in 0. In our notation the results of [S] correspond to choosing pi = po 
and Aj = Aq for all i, where |Ao — po| measures the degree of inconsistency. The resulting 
phase diagram is shown in Fig. |2l and corresponds to that derived in 01 by different 
methods. The dependence only on the combination |Ao — Pol reflects the equivalence of 
strategy biases and comfort levels in the homogeneous case. Without loss of generality, 
we will (mostly) consider the case po = 0. At any Aq one then finds a phase transition 
of the type which is observed in the standard MG, with an ergodic phase at a > ac(Ao), 
and non-ergodic behaviour below etc. In the ergodic regime time-translation invariance 
is maintained and the integrated response x remains finite. The phase transition line in 
Fig. 121 is marked by x ^ 00 (note that in the case of homogeneous strategy biases x' is 
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a multiple of x so that a divergence of x' is equivalent to one of x)- 

It is here interesting to study the convergence of the mean attendance (A) to the 
mean comfort level Aq. As shown in one finds that indeed (A) = Aq below etc, but 
not above (if Aq 7^ 0, for Aq = one always has (A) = by symmetry). In other words 
at fixed a agents are able to recover the mean comfort level even in the inconsistent 
case, provided strategy biases and comfort levels are uniform and the inconsistency 
smaller than some critical value Aoc(a)- This is illustrated in the left panel of Fig. 
0] (lower curves). At the same time the transition between the phases in which the 
mean comfort level can be retrieved, and the one where it cannot, coincides with the 
ergodicity-breaking transition. At fixed Aq the system is ergodic for a > adXo), and 
non-ergodic for a < ac(Ao), with large volatilities for unbiased starts at low a, and 
small volatilities for biased starts below the transition. Measurements of the volatility 
for unbiased starts are shown in Fig. 01 (right panel, upper curves). 

We now again turn to the effects of heterogeneity on this system. In the present 
case of inconsistent strategies one has to distinguish heterogeneities in the comfort levels 
Aj from those in the strategy biases pi . As we will see below strategy biases and comfort 
levels are no longer equivalent in the heterogeneous case, and it makes a crucial difference 
whether heterogeneities are added to one or the other. In order to disentangle the effects 
of both we study the following cases: 

(i) heterogeneous strategy biases at homogeneous comfort level {pi = ±0.05 randomly 
and Aj = 0.5, diamonds in Fig. EJ 

(ii) homogeneous strategy biases at heterogeneous comfort levels (p.j = and Aj = 
0.5 ± 0.05, circles in Fig. Oj) 

(iii) strategy biases and comfort levels both heterogeneous (p, = ±0.05 and A, = 
0.5 ± 0.05 with no correlation between pi and Aj, squares in Fig. IH} 

(iv) strategy bias and comfort level both homogeneous {pi = 0, Aj = 0.5, triangles in 
Fig. ED 

(iv) is the case of homogeneous comfort levels and strategy biases as discussed above. 
One finds that the transition as observed in (iv) is preserved only in case (i) where 
comfort levels are homogeneous, but that it is absent in cases (ii) and (iii) where comfort 
levels are heterogeneous. We illustrate this by plotting the mean attendance in the 
stationary states for all cases in the left panel of Fig. El While the mean comfort 
level Aq = 0.5 is successfully retrieved at low a in (i), systematic deviations are present 
irrespectively of a in (ii) and (iii). The right panel of the figure demonstrates that the 
volatility remains low in cases (ii) and (iii) (heterogeneous comfort levels) at low a for 
unbiased starts whereas diverges as a ^ for the systems (i) and (iv) where the 
comfort level is uniform. Note also the good agreement of the numerical experiments 
with the analytical theory in the ergodic regimes, up to small deviations due to finite- 
size and equilibration effects. Finally we also report simulations of a system with 
homogeneous comfort levels and strategy biases, but shifted by an amount A = —0.5 
with respect to (iv), i.e. we choose pi = po = —0.5 and Aj = Aq = for all i (stars 
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Figure 3. (Colour on-line) Deviation of the mean attendance (A) from the mean 
comfort level Aq (left), and fluctuations of the attendance (right). Solid lines are from 
theory, markers from simulations, unbiased starts, N = 1000 agents, run for 10^ steps, 
averaged over 10 samples. Triangles refer to a system with uniform strategy bias and 
uniform comfort level {pi = 0, Xi = 0.5), stars to the shifted case pi = — 0.5,Ai = 0. 
Diamonds, circles and squares represent cases (i), (ii) and (iii) as detailed in the main 
text. Vertical lines mark the phase transition for the model with uniform comfort level 
Xi = Xq = 0.5 as found from the analytical theory. 

in Fig. inj. Results are identical to those of (iv), confirming the translation invariance 
against simultaneous uniform shifts of the comfort levels and strategy biases. 

4.3. Fluctuating comfort levels 

Finally, we consider comfort levels which fluctuate in time in this section. Time- 
dependent comfort levels present a form of disorder which is not quenched so that 
the standard analytical tools are not straightforwardly applicable to this case. All 
results presented in this section therefore rely on numerical simulations of the on- 
line process (^. Fluctuating comfort levels are here implemented by drawing each 
Xi = Xi{t) from a bimodal distribution at every time step with no correlations in time 
(Ai(t)Aj(t') — Xi{t) Xi{t') = 0). We here distinguish between collectively and individually 
fluctuating comfort levels, 

collectively fluctuating: Aj(t) = Aq + s({t), 
individually fluctuating: Xi{t) = Aq + edit). 

Here ({t) and the (i{t) are drawn at random at every time step from the set { — 1, 1} with 
equal probability, with no correlations over time (or between players in the individually 
fluctuating case). All strategy tables are generated at the beginning of the game, and 
remain fixed thereafter. We limit the discussion to the case of vanishing strategy biases 
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- thGory fixed helorogengous comfort levels e=1 
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Figure 4. (Colour on-line). Game with resource levels fluctuating in time. Left: 
strategies consistent on average, right: inconsistent (see text for details). Simulations 
are for TV = 300 agents in the consistent case and N = 1000 agents for inconsistent 
strategy assignments. Solid lines in both panels are for comparison only, and show 
the theoretical lines for heterogeneous comfort levels at same spread e, but fixed in 
time. Filled triangles are simulations of the model with time-independent homogeneous 
comfort level. 



(pi = for all i) and study models with consistent and with inconsistent strategies. 
Consistency here refers to consistency as a time-average, i.e. to Aq = (Aj(t)) = po = 
for the consistent case and to Aq 7^ otherwise. 

Results are shown in Fig. 0] We will discuss the cases of collectively and individually 
fluctuating comfort levels separately in the following, and focus on the volatility of the 
attendance. 

4-3.1. Collectively fluctuating comfort levels: the corresponding volatilities are 
indicated as solid circles and squares in Fig. HI with the consistent case shown in the 
left panel, and the inconsistent case in the right one. We show results for two different 
magnitudes e in each case. For comparison we also display the analytical curves for fixed 
comfort levels (at the same degree of inconsistency as the simulations with fluctuating 
levels) as triangles in both panels. One thus concludes that a collectively fluctuating 
comfort level does not cause any noticeable effect on the resulting volatility of the 
system, but that the system instead behaves as if the collective comfort level was fixed 
at its time-average. 

4.3.2. Individually fluctuating comfort levels: the case of individually fluctuating 
comfort levels appears to be crucially different from the game with collectively 
modulated comfort level. Results are displayed as open markers in Fig. 01 and one 
finds that individually fluctuating comfort levels reduce the volatility significantly at 
small and intermediate values of a as compared to the system with homogeneously fixed 
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or collectively fluctuating levels. At larger values of a no such effect is found. While 
we cannot fully control equilibration in our experiments, due to apparently increased 
relaxation times, our simulations seem to suggest that individually fluctuating levels 
reduce the volatility below the one observed in a system with fixed heterogeneous 
comfort levels of the same spread e (indicated as lines in Fig. |3] for comparison). In 
particular we find that after sufficiently long e-dependent waiting times, the observed 
volatility shows only little sensitivity to the numerical value of e. 

5. Geometrical interpretation of the phase transition 

Our findings regarding the absence of the phase transition in El-Farol models with 
heterogeneous comfort levels is interesting also from the point of view of statistical 
mechanics of the MG. In this final section we will discuss a geometrical interpretation 
of the phase transition of the original MG and will make some suggestions of how it 
may be possible to approach the problem of characterising MG models according to the 
presence or absence of the phase transition marked by a diverging integrated response. 

The update rules of most known versions of the MG are of the form qi{t + 1) = 
Qiit) ~^ ^i^^^ i^^ on-line formulation), with some global time-dependent 
quantity A'^'^*)[q(t)] which depends on the information pattern n{t) presented to the 
agents at t and on the strategy score differences q(t) = (gi(t), . . . , qN(t)) of all agents. 
A geometrical picture behind the transition of the standard MG can here be devised 
as follows [inilH]: due to the above update rule, the A^- dimensional vector q(t) will 
move in the space spanned by the aN vectors = (^^, . . . , ^^), /i = 1, . . . , aN. We 
will abbreviate this space by VaN{{^}) = {S)^=i c^^'^ki, • • • , CaN ^ ^} in the following. 
Since a number (pN of agents can generally been shown to 'freeze' in MGs (where the 
fraction of frozen agents (p = 4>{a) depends on the details of the model), i.e. to employ 
one strategy only and to have \qi{t)\ ^ oo in the long-time limit, the effective number 
of degrees of freedom is reduced to [1 — (j){a)]N, so that q(t) can be thought of as having 
[1 — (f){a)]N free components and moving in the aA^-dimensional vector space VaN{{$,})- 
If a > 1 — (/>(«) then any perturbation on q(t) can be washed out by the dynamics. 
If however a < 1 — (f){a) this may not be the case (as the dynamics is restricted to 
movements in KtAr({^})), and initial conditions may become relevant. The point at 
which ergodicity breaking occurs (and the integrated response diverges) can thus be 
identified as ac = I — 0(ac), which is indeed fulfilled at the transition point of the 
standard MG, as illustrated in the inset of Fig. El 

This picture breaks down whenever the update rules are not of a form which moves 
the vector q(t) in the space spanned by the {^^}, specifically if q(t + 1) — q(t) is 
not a linear combination of the {^^}, and one might not necessarily expect to see a 
transition with diverging integrated response in this case. Several examples can be 
listed here. In grand-canonical MGs jTHI one has update rules of the (schematic) form 
qi{t + l) —qi{t) = — ^^^'•*''y4^(*) — K, with a model parameter k, and the transition is absent 
as soon as /t 7^ 0. MGs with finite score memory ^T] have update rules of the form 




Figure 5. (Colour on-line) Fraction of frozen agents (/> as a function of a. Main panel: 
model with consistent strategies, and heterogeneous fixed comfort levels {pi = Vi 
and Xi — ±£ randomly with e = 1, 0.5, 0.2, 0.01 from top to bottom. Symbols are from 
simulations of the batch process {N — 300 agents, averages over 10 samples, run for 
a minimum of 1000 batch steps, and longer if required for equilibration). Solid lines 
are predictions of the analytical theory. Inset: standard MG (e = 0), open symbols 
represent simulations with tabula rasa initial conditions, full symbols biased starts. 
Vertical line in the inset marks ckc where Ofc = 1 — 0(ckc)- 



qi{t + 1) — qi{t) = —'jqi{t)—^'j^^^^A^^^'^ (with < 7 < 1 defining the inverse time-scale over 
which scores are forgotten), and again the transition seems absent as soon as 7 > 0. No 
analytical results are available for games with finite score memory though. Finally the 
transition marked by a diverging integrated response is also absent in MGs with impact 
correction and with dilution, but instead preceded by a memory-onset transition (at 
which X remains finite). In both cases one has qi{t + 1) — qi{t) = — ^*''y4f^*'' where 
Af^^'^ carries an explicit dependence on i, hence invalidating the above picture of q(t) 
moving in VaNi{^})- Specifically Af = A^^{t) — (z// ^/N)^'j'si{t) in the model with impact 
correction (with u measuring the ability of the agents to correct for their own impact), 
and (t) = iV~^/2 J2j{cij/c){uj'^+^'^Sj{t)} in the model with dilution (where dj e {0, 1} 
and P{cij = 1) = c). 

In the model studied in this paper model one has A^ = A^^ — Aj with Aj the comfort 
level of player i. Thus the quantity Ai does not depend on i if Aj = Aq and the above 
geometric interpretation holds and one is not surprised to find the typical MG-transition. 
For heterogeneous {Aj} this is no longer the case, the picture of the vector q(t) moving 
in VaN{.{$,}) breaks down and the transition is absent. 

Having the listed examples in mind, one may thus speculate whether the property of 
the update rules of the standard MG of restricting the increments of q(t) to V^Ar({^}) is 
a necessary condition for a phase transition with diverging integrated response to occur. 
Note that such a dynamics would probably not be a sufficient condition for this type of 
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transition to be present as additionally the condition a = 1 — 4>{a) has to be met at ac- 
Interestingly we find that in our model with heterogeneous comfort levels this condition 
is never fulfilled, as 0(a) > 1 — a for all tested values of a (see Fig. E}. Only as the 
degree of heterogeneity e approaches zero we do find that 0(a) 1 — a for a < ac 
(where ac is the transition point of the MG with uniform comfort level, e = 0). Similar 
observations can be made for the GCMG. In order to test our hypothesis further, one 
may want to look for a model in which the dynamics is not restricted to VoAr({^}), 
but in which there is an a so that 1 — 0(a) = a. The MG with finite score memory 
might here be a candidate as no frozen agents are present at any a after long-enough 
equilibration jTT]. Unfortunately no analytical information on its phase behaviour is 
available at present. 

6. Concluding remarks 

We have investigated a mathematical formulation of the El-Farol bar problem focusing 
on the effects of heterogeneity in the comfort levels and/or the biases of the agents' 
strategies. Generalizing earlier results of jH] our main finding consists in the observation 
that heterogeneity in the comfort levels even of infinitesimal degree removes the phase 
transition of the standard MG, while no such removal is observed for heterogeneous 
strategy biases. This transition has also been shown to be present in an El-Farol bar 
problem with homogeneous comfort level. There one finds a phase at low but non-zero 
inconsistency of the strategies, in which the mean attendance converges to the comfort 
level, and a second phase in which the comfort level cannot be retrieved due to too large 
an inconsistency in the strategy vectors. In the phase of successful retrieval fiuctuations 
of the attendance around the comfort level are large (for tabula rasa starts), but depend 
on initial conditions. In the model with heterogeneous (i.e. agent- dependent) comfort 
levels the attendance converges to the mean comfort level if and only if strategies are 
fully consistent. This property is lost in the presence of heterogeneous comfort levels for 
any degree of inconsistency at any finite a. At the same time models with heterogeneous 
comfort levels appear to show low fiuctuations of the attendance, and the typical high- 
volatility branches of the model with homogeneous levels are absent. Most interestingly 
heterogeneous strategy biases at uniform comfort levels do not have the same effect, the 
transition is preserved. Hence we can strictly trace removal of the transition back to the 
spread in comfort levels, and find that the equivalence of comfort levels and strategy 
biases of the homogeneous case does not carry over to the heterogeneous one. 

We have also studied El-Farol games with temporally fiuctuating comfort levels 
and find that the behaviour of the volatility remains unchanged if the level fiuctuates 
collectively for all players. Individually fiuctuating levels, however, can reduce the 
volatility significantly. 

Finally we have pointed out the similarities of the present model with other MG- 
type systems with and without phase transitions, and have discussed a geometrical 
interpretation of the updated rules, which may allow to characterise MG-models 
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according to the presence or otherwise of the ergodicity breaking transition marked by 
a singular integrated response. We hope that these suggestions may stimulate further 
investigation of such models with a focus on their classification according to the types 
of their respective phase transitions. 
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Appendix: Equations describing the ergodic stationary states 

Assuming time-translation invariance (i.e. Cu' = C{t — t') and similarly for Gu and 
G^^i) and finite integrated response one follows the standard ansatz to proceed from the 
effective agent problem to explicit equations characterising the relevant persistent order 
parameters of the ergodic stationary states. In our problem these are given by c, the 
persistent part of the correlation function, and by x = YIit^^^) ^ ~ Ylir^'^'^)- 
Further details on this analysis relying on a separation of so-called fickle and so-called 
frozen agents can be found in [2]. 
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The resulting 3x3 system of non-linear equations for {c, x, x'} then reads 

dX i?(A)c(A), x = [dX RiX) x(A), x'= [dX i?(A)A x(A) (16) 



with -R(A) the distribution from which the strategy biases Aj are drawn and where x(A) 
and c(A) given by 
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and where g{X) is the persistent part of the temporal correlations of the noise rix{t) in 
the effective agent problem which reads 
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Following [2^ the fluctuations of the attendance can be approximated as 
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and the mean attendance level turns out to be 
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